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Abstract 

The purpose of this mixed methods study was to determine the impact of synchronous online tutoring 
services on struggling middle school students’ mathematics achievement. The online tutoring was 
provided as a response to intervention (RTI) Tier 3 support (intensive, individualized intervention) in 
schools implementing a school-wide mathematics program that addresses Tier 1 (high-quality classroom 
instruction) and Tier 2 (small group interventions). We employed quasi-experimental, within- and 
between-group designs to examine impacts for 119 students in two schools to measure the tutoring’s 
impact on mathematics assessment scores. We also conducted qualitative analyses of student and tutor 
postsession commentary. The findings suggest that the tutoring contributed to statistically significant 
gains in student assessment scores postintervention. Online tutors’ descriptions of their practice centered 
on ongoing progress monitoring of student learning, delivery of guided practice to students, the use of 
multiple explanations and representations of target concepts. Student perceptions of the online tutoring 
were predominately positive in nature. 


Introduction 

This article examines the impact of an online tutoring program on struggling middle school 
students’ mathematics achievement, tutor descriptions of their online tutoring practices, and student 
perceptions of tutoring experiences as they occurred in real time. The tutoring was provided as a Tier 3 
support to enhance the overall effectiveness of a school-wide mathematics reform effort, PowerTeaching 
math (PTm), a program funded under the U.S. Department of Education’s Investing in Innovation scale- 
up initiative. This initiative is focused on the dissemination of research-based educational programming to 
high-need schools. PTm is grounded in research on cooperative learning, and focuses on teamwork 
among students within a structured cycle of instruction (Slavin, 1995). There is an extant body of research 
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showing positive achievement effects for the instructional program, particularly among adolescent 
students (Nunnery, Chappell, & Arnold, 2013). Within a response to intervention (RTI) framework, PTm 
embeds Tier 1 and Tier 2 supports through the provision of professional development and resources that 
focus on specific elements of primary and secondary prevention of learning difficulties, including (1) 
effective instructional strategies, (2) differentiation, (3) strategic grouping of students, and (4) progress 
monitoring (O’Connor & Sanchez, 2011). Online mathematics tutoring was piloted in this context as a 
Tier 3 intervention for a subset of students who struggled the most with mathematics. 

Fuchs et al. (2011) identify instructional explicitness as the first principle of effective intervention 
for students with mathematics difficulties. Schema theory suggests that learners develop recognition of 
mathematical problems across numerous experiences to abstract problem-solving strategies that may be 
generalized (Brown, Campione, Webber, & McGilly, 1992). In other words, they develop schemas, or 
categories of problems that encompass similar types, and learn to apply increasingly effective strategies 
according to type. While typically developing students are able to do this more inductively, students who 
have accrued math deficits benefit from more explicit instruction facilitated by an expert who is able to 
provide clearly articulated schema-broadening instruction, with guided opportunities to recognize and 
learn the underlying mathematical structure of various problem types and specific transfer features (e.g., 
combinations of problem types) (Cooper & Sweller, 1987; Fuchs et al., 2011; Kroesbergen & Van Luit, 
2003). Theoretically, individualized mathematics tutoring should provide increased opportunity for 
instructional explication, leading to broadened schema among students who are having difficulty 
constructing these independently. Individual mathematics tutoring as a Tier 3 intervention also provides 
students with mathematics difficulties with increased individualized progress monitoring. No method 
works for all students all the time; students who struggle with mathematics even within the context of an 
instructional environment that applies validated, research-based methods thus require individual tailoring. 
In these cases, systematic, ongoing progress monitoring facilitates the formulation of adjusted 
interventions that are effective for those students (Fuchs et al., 2011). 

Although common characteristics of effective tutoring have been identified in the literature 
(Fashola, 1998), much of the focus has been on literacy rather than mathematics. With many school 
districts and funding agencies requiring research-based evidence on the effectiveness of strategies, the 
need for sound research on mathematics tutoring more specifically is clear. One earlier analysis of 
educational outcomes of mathematics tutoring revealed relatively large effect sizes in improving student 
achievement (Cohen, Kulik, & Kulik, 1982). Huang (2013) compared tutoring influences on performance 
across countries, controlling for social and educational differences among them, and found that tutoring 
significantly increased national mean perfonnance in mathematics. However, other researchers have 
concluded that not enough is known about the effect of tutoring on mathematics scores (Ritter, Barnett, 
Denny, & Albin, 2009; Baker, Rieg, & Clendaniel, 2006). Furthermore, some studies have found 
promising evidence to suggest that interactive online mathematics tutoring in particular may result in 
learning and achievement benefits for students (Beal, Walles, Arroyo, & Woolf, 2007; Chappell, 
Nunnery, Pribesh, & Hager, 2011; Nguyen & Kulm, 2005), but less is known about how these programs 
function as Tier 3 interventions in contexts that carefully structure Tier 1 and 2 supports. This study 
explores how online mathematics tutoring impacted student achievement outcomes and perceptions in the 
context of a wider reform that provided primary and secondary supports, and functioned to augment that 
effort as a tertiary intervention. 


Literature Review 


Mathematics Tutoring 

Instructional tutoring has an extensive history in American education. Tutoring services may be 
presented in a variety of formats and serve an array of purposes, though the overall goal of tutoring is 
generally to improve student achievement. The body of research on tutoring services is extensive and 
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reported results of tutoring vary widely (Chappell et al., 2011). Research findings about the effectiveness 
of tutoring for improving mathematics performance have been similarly inconsistent (Bray, 2011; Huang, 
2013; Rothman & Henderson, 2011). 

Although no specific model of effective supplemental tutoring has emerged from the literature to 
date, a number of successful programs have been identified, and these share some common 
characteristics, including well-trained, focused tutors; one-to-one tutoring experiences; a structured but 
flexible prescribed instructional plan; ongoing assessment to inform instructional efforts (Fuchs et al., 
2011); and consistency in receiving tutoring services (Fashola, 1998; Wasik, 1998; Topping, 2000; 
Gordon, 2003; Sanderson, 2003). Fashola (1998) summarized the issue concisely: Programs that “provide 
greater structure, a stronger link to the school-day curriculum, well-qualified and well-trained staff, and 
opportunities for one-to-one tutoring seem particularly promising” (p. 55). 

Other research offers additional insight about structural elements and tutor characteristics for 
effective mathematics tutoring programs. Effective mathematics tutors utilize their knowledge about 
mathematics content and curriculum when tutoring to generate multiple explanations and representations 
of the content for and with students, which is required for the development of mathematical understanding 
(Bray, 1999; Charalambos, Hill, & Ball, 2011; Hiebert et al., 1997; Rothman & Henderson, 2011). 
Effective tutors also know when and how to press learners, what resources to use, and how to use 
strategic questioning in lieu of telling or giving answers (Donnelly, 2013). Additionally, mathematics 
tutors who cultivate relationships with students and their teachers are more effective because in-school 
instructional activities and tutoring can be designed synergistically to build upon each other (Bums, 
Senesac, & Symington, 2003; Rothman & Henderson, 2011). 

Online Tutoring 

Instructional technology is becoming more and more prominent in educational settings, 
expanding beyond industry and higher education into K-12 environments. A systematic review conducted 
by Kulik (2003) indicated that instructional technology was becoming not only prominent in K-12 
education but also a progressively more effective instructional strategy. Programs that offer online 
services may be able to provide benefits beyond just instructional support. For example, online tutoring 
programs can combine effective features of traditional tutoring services, such as research-based curricular 
resources and well-trained, qualified tutors, with synchronous supplemental instruction that can meet 
students where they are—academically and physically (Vasquez & Slocum, 2012). Online, synchronous 
services that offer one-to-one, real-time tutoring can reach students in busy, urban areas as well as 
isolated, rural schools (Koerwer, 2007). 

Over the past decade, online mathematics tutoring has emerged as an effective strategy to support 
student achievement in mathematics. A meta-analysis of out-of-school-time tutoring programs indicated 
that those agencies offering online mathematics tutoring had a larger effect size than agencies offering 
only face-to-face mathematics services (Chappell et al., 2011). A study by Beal et al. (2007) revealed that 
students showed significant improvement on mathematics problems after receiving online, interactive 
tutoring. The gains were experienced only in skills for which students received tutoring, as Beal et al. 
(2007) included problems with nontutored skills as control items in their analyses. These findings support 
other promising results that suggest that students benefit from online, real-time mathematics tutoring 
(Carnegie Learning, 2002; Nguyen & Kulm, 2005). More importantly for the current study, results from 
such studies have revealed that students struggling with mathematics achievement benefit the most from 
online mathematics tutoring (Maloy, Razzaq, & Edwards, 2014; Beal et al., 2007). 

Research indicates that online mathematics tutoring should be consistent in order for participants 
to experience significant gains in achievement. However, the time investment required to achieve these 
gains is not substantial. Clark and Whetstone (2014) found that students who spent “as little as 50 minutes 
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per week” (p. 465) participating in online mathematics tutoring experienced an effect size of .68— 
equivalent to more than two thirds of a year’s worth of improvement. Even more impressive are the 
results from a study of middle school students that found that more intensive tutoring at five hours per 
week for a 6-week period resulted in an average gain of an entire grade level in mathematics for 
participating students (eSchool News, 2009). In addition to employing tried-and-true, research-based 
traditional tutoring strategies, most online mathematics tutoring efforts also take place during the school 
day, often in a pull-out approach—a reality that baffles out-of-school-time tutoring proponents but has 
proven effective nonetheless. 

Relevance of the Study and Research Questions 

This study adds to the body of knowledge surrounding online mathematics tutoring by exploring 
how outcomes of tutored middle school students compare to those of students exposed only to their 
schools’ business-as-usual mathematics instruction. The results are intended to provide sound, statistical 
evidence of the impact of mathematics tutoring, a need identified in the literature (Ritter et al., 2009, 
Baker et ah, 2006). We addressed the following research questions in this study: (1) How does online 
mathematics tutoring affect mathematics scores of low achieving students? (2) What are students’ 
perceptions of online mathematics tutoring services? (3) How do online mathematics tutors describe their 
pedagogical practices and student learning? 


Methods 

We employed a mixed methods approach, using quantitative methods to address the tutoring’s 
impact on mathematics scores and qualitative methods to examine tutor and student commentary on the 
services. We used a quasi-experimental, pretest/posttest design (Leedy & Ormrod, 2010) to analyze 
within-group changes in achievement scores for participating students, with separate analyses conducted 
for each school. Additionally, we conducted a causal comparative matched sample design to compare 
tutored students’ achievement to that of nontutored students in School 1. Model-guided and inductive- 
coding qualitative techniques were used to analyze tutor descriptions of their online tutoring practice and 
student perceptions of the program (Zhang & Wildemuth, 2009). 

Participants 

Students from two middle schools participated in this study. School 1 is a large rural school in 
southern Virginia, and School 2 is a large rural school in central Kansas. Both schools were also serving 
as pilot schools for a school-wide mathematics program, PTm, as part of an Investing in Innovation scale- 
up grant through the U. S. Department of Education. The tutoring occurred during Year 2 of the 5-year 
program. 

All student participants in the study, both treatment and comparison, earned below passing scores 
on either the state standardized mathematics assessment for the 2012-2013 academic year (School 1), or 
failed the program’s pretest assessment at the beginning of the 2013-2014 academic year (School 2). 
Forty-nine Grade 6 students from School 1 and 70 Grade 7 and 8 students from School 2 received 
tutoring during the 2013-2014 academic year. Students began receiving services in October 2013 and 
attended tutoring twice a week for 20 weeks. An additional 292 Grade 6 students in School 1 who did not 
receive tutoring services were included in the between-group analyses for that school. 

Attendance records indicated that participants in School 1 attended an average of 28 sessions that 
were 30 minutes in length. On average, School 1 students received 14 hours of tutoring. Sixty tutors 
provided services for these students, with students working with six tutors on average throughout the 
program. Participants in School 2 attended an average of 38 sessions lasting 37 minutes per session, or 
about 23 hours of tutoring per student. A total of 61 tutors provided services for these students, with 
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students working with an average of 8 tutors over the duration of the program. Tutoring services were 
provided twice a week in pull-out fashion during normal math class time for participating students. 

All students in both schools were exposed to the school-wide mathematics program, PTm. In 
School 1, all students also received remediation in core subjects in a tiered RTI style. No other services 
were offered to nontutored students in School 1. In School 2, only those nontutored students with 
identified disabilities received additional mathematics services; other nontutored students did not receive 
any additional instruction in mathematics. 

Description of Services 

For this study, online mathematics tutoring was provided by Focus EduVation (FEV), a program 
whose services include interactive tutoring for K-12 students. Tutoring was provided in a synchronous, 
one-to-one online environment using chat, instant messaging, and virtual whiteboard technology. 
Grounded in principles of effective remediation for students with mathematics difficulties, the program 
focuses on individualized progress monitoring to design and adjust tailored interventions (Fuchs et al., 
2011). Diagnostic assessments were administered to participating students before tutoring began to allow 
program and school personnel to develop individualized learning objectives for each student. Based on 
these learning objectives, a learning plan was created for each student. Learning plans were aligned with 
the school’s curricular standards and scope and sequence to ensure that the services met the student’s and 
school’s specific needs. The learning plans served as the foundation for the tutoring but were flexible 
enough to accommodate minor changes over the duration of the program. Overall, the tutoring was 
intended to provide a differentiated, engaging environment where skills were enhanced through sharing of 
curricular materials, practice problems and visuals, and graphic features to aid communication and 
collaboration between students and tutors. Shared communication was focused on providing students who 
struggle with mathematics with expert instructional explication to promote lateral transfer and schema 
development (Brown et al., 1992; Gick & Holyoake, 1983). Tutors providing services were required to 
have a minimum of a 4-year degree and 2 years teaching or tutoring experience, and underwent 
background checks to ensure a safe environment. In-session assessments allowed tutors to assess student 
progress by specific skill in a formative manner, and posttests allowed for an examination of students’ 
overall growth. 

Data Sources 

We used scores from the 2013 and 2014 administrations of the Virginia Standards of Learning 
(SOL) assessment for participating students as pretest and posttest for School 1 and program-administered 
pretests and posttests for School 2 (state-level assessments were unavailable for School 2 due to a 
moratorium on testing in 2014). 

Additionally, we examined tutor-generated summaries of each online tutoring session, as well as 
responses to open-ended items from students that solicited their perceptions of the tutoring services. 
These summaries and responses were used to explore themes related to tutor practices and student 
perceptions of the tutoring they received. These data were collected by the tutoring agency and provided 
to us through a data use authorization with the two schools. 

Analytic Approach 

We conducted paired samples /-tests for each school using pretreatment and posttreatment scores 
to determine within-group changes in each school. Tutored students in School 1 were also matched with 
nontutored students (see matching procedures below), and between-group differences were examined 
using an analysis of covariance (ANCOVA). We also computed Cohen’s d within-group effect-size 
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estimates for tutored and comparison students using the correction for dependence method (Morris & 
Deshon, 2002). 

Matching procedures. The propensity score matching approach (Guo & Fraser, 2010) was used 
to match the Grade 6 tutored students in School 1 with nontutored students. A total of 407 students were 
enrolled in Grade 6 mathematics and were included in the initial matching procedures. Sixty-six students 
were removed from the sample because they did not have achievement scores for both 2013 and 2014. 
Ultimately, 341 students were included in the matching procedures, with 49 in the treatment group and 
292 in the comparison group. 

Propensity scores were estimated by the predicted probabilities of a logistic regression model 
(Guo & Fraser, 2010) with the tutoring status as the criterion variable and the following covariates 
serving as predictor variables: minority status, economically disadvantaged status, disability status, 
student in recovery status, gender, and 2013 SOL scaled score. Student demographics were those assigned 
by the division/state. Descriptions are as follows: minority status, all students not categorized as White 
(Black or African American, Flispanic, and multiracial); economically disadvantaged, students eligible for 
free or reduced-price lunch; disability status, any student identified with a learning, physical, emotional, 
behavioral, or mental disability; and student in recovery status, any student who has participated in a 
remediation program. 

The overall model fit of the logistic model was supported, j 2 (8, N= 341) = 14.76, = .07. The 1- 
to-2 nearest neighbor matching approach (Guo & Fraser, 2010) was utilized to match each tutored student 
with two nontutored students with the smallest absolute differences of the estimated propensity scores. As 
a result, 49 tutored students were matched with 98 nontutored students. 

Prior to the matching procedure, the tutored and the nontutored groups differed significantly on 
the minority, economically disadvantaged, recovery, and 2013 achievement scores predictors (see Table 

1) . After the matching procedure, the tutored students were statistically equivalent to the nontutored 
students on all covariates except the recovery predictor and the baseline achievement score, with the 
nontutored group having more students in recovery and a higher mean 2013 SOL scaled score (see Table 

2) . The overall group equivalencies improved with the propensity score matching, and we proceeded with 
the analyses using the recovery indicator and the 2013 mean SOL scaled score as covariates in our 
ANCOVA to control for the pretreatment differences. 


Table 1 Covariate Descriptive Statistics and Equivalency Checks by Tutoring Status Before Matching (n = 341) 


Covariate 

Tutored 
(overall n = 49) 

Nontutored 
(overall n = 292) 

Equivalency 

analysis 

Minority 

32 

120 

* 2 = 9.96* 

Economically disadvantaged 

37 

161 

X 2 = 7.15* 

Disability 

8 

43 

* 2 = .09 

Recovery 

32 

63 

* 2 = 39.93** 

Female 

27 

155 

X 2 = -07 


49 

292 


2013 SOL Scaled Score 

(m = 383.08) 

(m = 468.97) 

t = 15.72** 


* Statistically significant at the .01 level 


** Statistically significant at p < .001 
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Table 2 Covariate Descriptive Statistics and Equivalency Checks by Tutoring Status After Matching (n = 147) 


Covariate 

Tutored 
(overall n = 49) 

Nontutored 
(overall n = 98) 

Equivalency 

analysis 

Minority 

32 

59 

* 2 = 0.36 

Economically disadvantaged 

37 

74 

z 2 = 0.00 

Disability 

8 

21 

X 2 = 0.54 

Recovery 

32 

44 

X 2 = 5.45* 

Female 

27 

45 

z 2 = 1.10 


49 

98 


2013 SOL Scaled Score 

(m = 383.08) 

(m = 406.90) 

t = 5.42** 


* Statistically significant at the .05 level 


** Statistically significant at < .001 

To address Research Questions 2 and 3, we analyzed 84 tutor-generated session summaries and 
84 postsession commentaries by students across both schools. These data were analyzed using NVivo, a 
qualitative data analysis software program that enables users to create a flexible node structure where 
similar data may be coded throughout analysis to assist in the iterative construction of categories and 
themes (Patton, 2002). Both model-guided and inductive coding were employed in the analysis (Zhang & 
Wildemuth, 2009). Analysis of tutor summaries of online sessions was guided by practice components of 
the tutoring model and the literature on pedagogical strategies for mathematics remediation. Student 
comments were analyzed to identify students’ emic perceptions of the program. The codebooks developed 
for analyses of tutor and student comments appear in Appendix A. 

Several strategies were employed to enhance the trustworthiness of findings emerging from this 
process: (a) two researchers analyzed each data source; (b) researchers met frequently to establish, clarify, 
and revise codes and categories; (c) an electronic audit trail was maintained; (d) a program model 
provided referential adequacy; and (e) findings were contextualized within the broader literature on 
mathematics learning and pedagogy (Shenton, 2004). These strategies extend the credibility, 
confirmability, transferability, and dependability of the findings. Further, representativeness indices were 
calculated to provide infonnation about the extent to which themes emerged across individuals, and how 
prevalent each was throughout the discourse. These indices provided context related to the number of 
sources with statements coded to each theme at least once, and the salience of themes in terms of the 
number of references generated across participants accruing to a single code. 

Results 


Within-Group Outcomes in Schools 1 and 2 

Scores on the program-administered pretests indicated that participating students in each school 
had below-passing averages. School 1 had no students with passing averages on pretest assessments and 
no students with passing scaled scores on the prior year’s mathematics SOL. School 2 also had no 
students with passing averages on the program administered pretest assessments. 

At the end of the program, 30 (61.2%) of the School 1 students had passing scaled scores on the 
mathematics SOL. Before exposure to the program, the mean score for participating students in School 1 
on the 2013 Virginia SOL mathematics assessment was 383.08, below the proficiency cut score. The 
average score on the Virginia 2014 SOL mathematics assessment for the same students in School 1 was 
405.96, which is above the 400 cut score for proficiency in Virginia (Virginia Department of Education, 
2012). Within-group differences from pre-intervention to postintervention for School 1 showed 
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significant improvement in scores for the group, with a mean improvement of 22.88 points (t = 5.99, p < 
.001). Within-group effect-size estimates for School 1 were d= +0.95 for tutored students. 

At the end of the program, 15 (30%) of the tutored students in School 2 had passing averages on 
program-administered posttests. Although the posttest average for School 2 was below the passing level 
(M = 53.03), students in School 2 did show significant improvement in achievement, with a mean gain of 
26.16 points (t = 10.11, p < .001). The within-group effect-size estimate for tutored students in School 2 
was d = +1.47. 

Table 3 reports pretreatment and posttreatment means, mean differences, and probability values 
for both schools. Figures 1 and 2 illustrate the gains in achievement for each school. 


Table 3 Paired-Samples Results for Mean Outcome Scores by School 



n 

Pretest 

mean 

(SD) 

Posttest 

mean 

(SD) 

Mean 

difference 

P 

School 1 

49 

383.08 

(11.31) 

405.96 

(26.36) 

22.88 

<.001 

School 2 

50 

26.87 

(11.68) 

53.03 

(16.76) 

26.16 

<.001 
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Figure 2. Comparison of pre- and posttreatment scores for students receiving services in School 2. 



Between-Group Outcomes in School 1 

The ANCOVA indicated nonsignificant differences between the 2014 SOL scaled posttest scores 
for the tutored group and the matched, nontutored group, F = 1.66 (1,144), p = .20. This means that the 
nontutored group’s statistically significantly higher baseline score advantage was narrowed 
postintervention, after controlling for recovery status and 2013 SOL scaled score. It is worth noting, 
however, that both groups had statistically significant within-group increases in scores from pretest to 
posttest, indicating significant growth for both groups (within-group results for comparison group: t = 
2.35, p = .02). Within-group effect-size estimates for School 1 were d = +0.95 for tutored students, and d 
= +0.24 for nontutored students. Table 4 reports between-group comparisons by treatment status. 


Table 4 Analysis of Covariance for Outcome Scores for School 1 



Pretest 
mean* (SD) 

Observed 
posttest 
mean (SD) 

Adjusted 

posttest F p 

mean (SE) 

Tutored students 

383.08 

405.96 

409.28 


(11.31) 

(26.36) 

(5.74) 




1.66 .20 


406.90 

424.20 

422.54 

Comparison students 

(40.48) 

(65.95) 

(8.26) 


* Pretest mean difference between tutored and comparison students was statistically significant at the .001 
level. 
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Tutoring Descriptions 

Tutor-session log data across both sites revealed that almost all tutors discussed implementing 
pre- and postassessments of student learning during each session, and analyzed these data to guide the 
instruction provided in the session. Many of the tutors also provided descriptions of their reflections on 
student learning during independent practice, and provided individualized differentiation as needed. 
Almost all tutors described the delivery of guided practice to students, and many discussed the use of 
multiple explanations and representations of target concepts. Tutors described interventions focused on 
accessing prior knowledge, modeling (“showed him how to multiply the decimal number with example”), 
explaining the steps in mathematical processes (“explained all the five steps to find the square root of the 
irrational numbers”), identifying both process and operational errors (“another problem of similar kind 
was worked by the student with minor mistakes, which she corrected on tutor’s advice”), and scaffolding 
through the use of questions or prompts (“tutor helped the student by providing hints to get the right 
answer”). 

Tutor reflections on student learning were prevalent throughout the summaries and included 
articulation of student’s postlesson content mastery, session scores, and identification of prior knowledge 
that was necessary for success. Identification of various types of student difficulties as well as leverage 
points for further instruction were often present, as exemplified by the following tutor comments: “The 
student struggled with multiplying the decimals, but corrected himself and was fast while learning,” and 
“Student was able to translate the given image and could reflect the image over different axis on the 
coordinate plane with ease.” 

Student Perceptions 

Analysis of student postsession commentary on the tutoring sessions revealed both positive and 
negative perceptions. Positive comments outnumbered negative comments by a ratio of three to one. Most 
of the positive student commentary was general in nature, expressing gratitude for the assistance, a 
fondness for the tutor, or social greetings. The general nature of student responses is not surprising given 
the age of students and the open-ended nature of the prompt to simply comment on the session. However, 
even within this unstructured environment, approximately 40% of students spontaneously indicated that 
they found the session to be helpful for their mathematics learning, and about a quarter of students 
identified specific learning outcomes of the session. For example, one student commented, “Good 
session—now 1 understand integers.” Another noted, “Now 1 get irrational numbers and rational 
numbers.” A smaller percentage of students made specific statements indicating they left the session 
feeling more confident in the material, as the following comments illustrate: “He made it seem so easy, 
because it is,” and “ft is easy once you learn it.” 

Student negative commentary about the tutoring sessions was far less frequent than positive 
commentary, but was often more specific in nature. The two most prevalent themes of negative response 
to tutoring were related to pacing of the sessions, and the desire for more explanation. Students who made 
comments about pacing were far more likely to say the session went too slow rather than too fast. Due to 
the general nature of much of the commentary, it was unclear whether this related primarily to the speed 
of connection or to the actual pedagogical pacing. However, some students did specifically articulate that 
too much time was spent on tutor explanation and not enough on student work. For example, one student 
noted, “He doesn’t let me work out the problem even when he tells me to solve it on my own.” Of 
students who did desire more explanation, there was a desire for greater clarity of explanation, as 
expressed by the following student: “This tutor was great, just needs to work on explaining a little bit 
more and showing his work more clearly.” 
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Findings and Discussion 

The findings of this study suggest that well-designed-and-implemented synchronous online 
mathematics tutoring may be an effective Tier 3 intervention for students in high-need settings who are 
struggling with mathematics. Our findings support the extant literature on online mathematics tutoring 
and provide further evidence that the strategy can be successful in improving mathematics achievement of 
underperforming middle school students when compared to their nontutored peers. Although the 
propensity score matched comparison analysis did not reveal a statistically significant difference in the 
adjusted means between tutored and nontutored students, within-group effect-size estimates indicated a 
large relative advantage for tutored students in School 1 (d = +0.95 for tutored versus d = +0.24 for 
nontutored students), and a large absolute gain for tutored students in School 2 (d = +1.47). It is worth 
noting that these results were observed in a context where the schools were implementing a well- 
resourced and proven instructional model that addresses Tier 1 and Tier 2 R.T1 supports. 

Tutor-described practices suggested good fidelity of implementation of the model and alignment 
with the literature. Almost all tutors discussed the use of formative assessments of student learning 
before, during, and after each session to guide the instruction provided in the session and set goals for 
future sessions. These findings are consistent with the goals of the program studied here as well as the 
literature on the benefits of systematic, ongoing progress monitoring as a core principle of effective 
remediation for students who struggle with mathematics (Fashola, 1998; Fuchs et al., 2011). Tutors 
described the delivery of guided practice to students, and many discussed the use of multiple explanations 
and representations of target concepts. These approaches described by tutors are consistent with literature 
in the field that links instructional explanations and representations with development of student 
mathematics understanding (Charalambos et al., 2011; Fuchs et al., 2011; Fliebert et al., 1997). 

These qualitative findings point to some of the mechanisms at work behind the achievement 
results and suggest some specific directions for practice in online mathematics tutoring as a Tier 3 
intervention. Specifically, these tutors described attending assiduously to student pre- and postdata to 
guide their instructional moves, and to providing multiple scaffolded opportunities for the students to 
engage with the mathematics at their individual levels. Session tutoring logs contained evidence of 
articulation of the student’s postlesson content mastery; identification of the student’s individual 
difficulties; descriptions of the student’s learning behaviors; reflection on the student’s mathematical 
understanding during the lesson; and recommendations for future teaching/leaming. Online tutoring 
programs might consider incoiporating specific prompts in each of these areas for tutors to respond to 
postsession, to help ensure all of these components are systematically addressed in each session. 

Student commentary further supported the use of online mathematics tutoring that is data driven, 
individualized, and focused as a Tier 3 intervention for students who struggle. Although a great deal of 
the student postsession commentary was general in nature, it was overwhelmingly positive. More 
importantly, even within the context of a fairly unstructured prompt, some students identified specific 
learning outcomes of the sessions. This suggests that online mathematics tutoring that embeds consistent 
progress monitoring may engage students in thinking about their own mathematics learning as a process 
for enhancing achievement. A possible implication for practice is to consider how to include specific 
prompts for students to co-reflect on their learning with their online tutors. For example, online tutoring 
sessions might conclude with a prompt for students to identify what they learned that session and to 
explain how they know they learned this, providing students with the opportunity to think about their own 
mathematical understanding and the tutors with a window into student thinking. 

Student perceptions of the pacing of online tutoring were mixed, which is possibly a function of 
the wide variety of reasons students may be assigned to a Tier 3 mathematics intervention. For example, 
for some students the achievement gap may be due to processing issues, and thus may require more 
explanation, while for others, attention-related issues may demand a faster pace. Furthermore, the 
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expressed desire on the part of at least some students for more time to work out problems on their own 
resonates with the literature that describes tutoring effectiveness as a function of knowing how to employ 
strategic questioning before telling or giving answers to students (Donnelly, 2013). Similarly, a number of 
students expressed the desire for more autonomy in the online tutoring process, from the desire to move 
faster or slower, to work more independently or with more support, and to select a preferred tutor for each 
session. These are ideas that are valuable to consider in developing online tutoring platforms, particularly 
for middle school students who are typically in a location along the developmental trajectory that makes 
choice and control of learning especially desirable (Midgley, Anderman & Hicks, 1995). 

It is also important to note that students participating in this study were among the lowest 
performing students in two high-need rural middle schools with large numbers of students living in 
poverty. These are among the most underrepresented students in advanced STEM study and careers 
(Avery, 2013; Eick et al., 2007; Suresh, 2011) and the least likely to possess the mathematics confidence 
that motivates sustained engagement, persistence, and learning in the discipline (Perez, Cromley, & 
Kaplan, 2014; Singh, Granville, & Dika, 2002). Yet close to half the students indicated that they believed 
they learned during the tutoring session, a quarter identified a specific learning outcome, and some made 
detailed comments directly expressing their own increased confidence in their ability to understand and 
learn math. These findings suggest that for this particular population of students, the highly individualized 
and explicit nature of the tutoring support may have resulted in affective changes that are related to 
mathematics engagement and learning. The online nature of the program made this support available to 
students in isolated rural communities with comparatively limited access to resources for providing this 
type of focused, individualized instruction. Exploration of this theme and the impact of online tutoring 
programs that embed explicit schema-broadening instruction and systematic progress monitoring on 
students who have not traditionally succeeded with mathematics nor been well represented in higher level 
STEM study and careers is an area for further research. 

Limitations 

It should be noted that the study did not randomly assign equivalent students to the treatment 
condition, so it is not possible to definitively attribute outcomes to the tutoring program. Further, the 
generalizability of the results is limited in that both samples were comprised of very low-performing 
students from rural populations, and both schools were already engaged in a substantial school-wide 
mathematics instruction improvement initiative. Notwithstanding these caveats, this study provides 
promising evidence that a well-designed-and-implemented synchronous online tutoring program can 
augment the effectiveness of a robust, research-proven classroom model of mathematics instruction. 
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Appendix A 


Table A1 


Codebook for Student Tutoring Session Comments 


Category 

Codes 

Examples 

Social 
interaction 
with tutor 

1. Statements expressing thanks 
for the session 

2. Statements expressing a general 
positive attribution to the 
session or tutor 

3. Social greetings 

1. “Thanks for working with me.” 

2. “1 love your teaching.” 

3. “Talk to you on Tuesday.” 

Learning 

benefits 

1. Statements indicating the 
student found the tutor to be 
helpful to them 

2. Statements that the student 
constructed a specific 
mathematical understanding 

3. Statements indicating student 
left session feeling confident 
about material 

4. Statements that the student liked 
the website or tools 

1. “He worked with me really well. He was nice 
and helped me a lot.” 

2. “Good session 1 now understand integers.” 

3. “Now I get irrational number and rational 
numbers.” 

4. “He made it seem so easy, because it is” 

5. “Hike the white board.” 

Learning 

challenges 

1. Pacing is too slow 

2. Pacing is too fast 

3. Desired more 
explanation/clarification 

4. Desired more active 
participation 

5. Problems with website and/or 
connectivity 

1. “Very slow, pick up the pace.” 

2. “1 feel like this is helping me but I also feel 
like my tutor rushes me.” 

3. “The tutor was great, just needs to work on 
explaining a little bit more, and showing his 
work more clearly.” 

4. “ft was good but he needs to let me do some 
work by myself.” 

5. “At one point the thing didn’t let me type 
that’s why 1 took so long.” 
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Table A2 


Codebook for Analysis of Online Math Tutor Logs 


Categories 

Codes 

Illustrative language 

Assessment of 
student’s prior 
knowledge 

1. Analysis of pretest data 

1. Posted the pretest questions for the student 
to solve. The test consisted of 10 problems. 
[Student] attempted 8 problems, and solved 

5 of them correctly. Student was not 
familiar with concept of Pythagorean 
theorem. 

Online tutoring 
interventions to 
support student 
understanding 

1. Accessing prior knowledge 

2. Modeling 

3. Explaining steps in 
mathematical processes 

4. Pointing out student mistakes to 
the student 

5. Providing scaffolding through 
questioning or prompts 

1. Started session by giving warm-up problem 
on dividing decimals with whole number. 

2. Showed him how to multiply the decimal 
number with example. 

3. Explained all the five steps to find the 
square root of the irrational numbers. 

4. Another problem of similar kind was 
worked by the student with minor mistakes, 
which she corrected on tutor’s advice. 

5. Tutor helped the student by providing hints 
to get the right answer. 

Tutor reflection 
on student 
learning 

1. Articulation of student’s 
postlesson content mastery 

2. Identification of student 
difficulties, mathematics 

3. Identification of technological 
issues impacting learning 

4. Description of student learning 
behaviors 

5. Reflection on student 
mathematical understanding 
during lesson 

6. Recommendations for future 
teaching/leaming 

1. Attempted eight problems and solved five of 
them correctly. He scored 50% in the 
session. 

2. The student faced difficulty in multiplying 
and adding the numbers. 

3. The student struggled with writing on the 
whiteboard. 

4. The student struggled with multiplying the 
decimals, but corrected himself and was fast 
while learning. 

5. Student was able to translate the given 
image and could reflect the image over 
different axis on the coordinate plane with 
ease. 

6. He needs more practice in multiplication of 
fractions. 
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